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In this paper we study a class of stochastic control problems in 
which the control of the jump size is essential. Such a model is a 
generalized version for various applied problems ranging from op- 
timal reinsurance selections for general insurance models to queue- 
ing theory. The main novel point of such a control problem is that 
by changing the jump size of the system, one essentially changes 
the type of the driving martingale. Such a feature does not seem 
to have been investigated in any existing stochastic control litera- 
ture. We shall first provide a rigorous theoretical foundation for the 
control problem by establishing an existence result for the multidi- 
mensional structure equation on a Wiener-Poisson space, given an 
arbitrary bounded jump size control process; and by providing an 
auxiliary counterexample showing the nonuniqueness for such solu- 
tions. Based on these theoretical results, we then formulate the con- 
trol problem and prove the Bellman principle, and derive the cor- 
responding Hamilton-Jacobi-Bellman (HJB) equation, which in this 
case is a mixed second-order partial differential/difference equation. 
Finally, we prove a uniqueness result for the viscosity solution of such 
an HJB equation. 



1. Introduction. In this paper we are interested in a class of stochastic 
control problems in which the dynamics of the controlled system take the 
following form: 

(1.1) Y t = x+ [ t b(Y s ,u s ,ir s )ds+ f a(Y s _,ir s ,u s )dX^ t>0, 
Jo Jo 
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where X u is a martingale that satisfies the so-called structure equation: 



(1.2) [X u ] t = t+ / u s dX^, t>0. 

Jo 

In the above [X u ] denotes the quadratic variation of X u , and u is some pre- 
dictable process. It is easily seen that the process u "controls" exactly the 
jumps of X u , whence that of Y. We note that a martingale X u satisfying 
the structure equation (1.2) must satisfy (X u ) t = t, that is, it is a so-called 
normal martingale (cf. Dellacherie, Maisonneuve and Meyer [8]). In fact, it 
is known that if a normal martingale X u has the so-called representation 
property, then it must satisfy the structure equation (see, e.g., Emery [10] or 
Section 2 for more detailed discussions). Typical examples of normal martin- 
gales satisfying structure equation include Brownian motion, compensated 
Poisson process and Azema martingale, etc. 

A stochastic control problem with control being the jump size can be seen 
from the following example, which more or less motivated our study. In an 
optimal reinsurance and investment selection problem, the dynamics of the 
risk reserve of the insurance company can be described, in general, by a 
stochastic differential equation (SDE) of the following form (see, e.g., [16]): 

Y t = y+ I b{Y s ,a s (-),ir s )ds + f a(ir 8 )dW 8 

(1.3) J ° J ° 



a s (x)f(s, x)N{dx, ds), 



where W is a standard <i-dimensional Brownian motion, representing the 

uncertainty of the underlying security market; St = Jo Jr f(s,x)N(dx,ds), 

with N being a compensated Poisson random measure, denotes the accumu- 
lated incoming claims up to time t; the random field a is the " reinsurance 
policy" (or "retention ratio") and tt = (it 1 , . . . , tt 11 ) is the usual investment 
portfolio. If we consider only those reinsurance policies a(-) for which there 
exist predictable processes (5 and u such that the following equation holds: 

(1.4) X?' u = f ' (5 s dW s + I I a s (x)f(s,x)N(dx,ds), 



10 JO 

where X 0,u satisfies the structure equation (1.2), then, noting that the 
Brownian motion W itself satisfies the structure equation with (tt = 0), 
we can then rewrite equation (1.3) as the general form of (1.1), in which 

X u = (W, X°' U ) T is now a (d+ l)-dimensional normal martingale. In fact, if 
the probability space is properly chosen, then one can show that (see Section 
2 for details) for any bounded predictable process u, there are always such 
a and (3. 



STOCHASTIC CONTROL FOR NORMAL MARTINGALES 



3 



We remark that the special nature of the normal martingale gives rise 
to a financial market model that is complete, but with jumps (cf. Dritschel 
and Protter [9]). This is based on an Ocone-Haussmann-Clark type formula 
for normal martingales established in Ma, Protter and San Martin [17]. We 
should also note that as a special class of normal martingales, the solutions 
to structure equations are interesting in their own right, and have been stud- 
ied by many authors (see, e.g., Attal and Emery [3, 4], Taviot [21] and, very 
recently, Emery [11]; for the two-dimensional case, see also Attal and Emery 
[3, 4] and Kurtz [14]). However, this is a subject that has not been explored 
fully. In fact, to our best knowledge, there has not been any general result re- 
garding the well-posedness of such equations in higher dimensional cases. In 
this paper we show that, in a Wiener-Poisson space, the issue becomes much 
more tractable. In fact, we shall derive a necessary and sufficient condition 
for a (multidimensional) normal martingale to satisfy a structure equation 
in such a space. This characterization theorem will then lead to the existence 
theorem of the (multidimensional) structure equation in a Wiener-Poisson 
space. An interesting observation, however, is that such a solution is not 
unique, even in the sense of law(!). We shall provide a counterexample in 
the Appendix for the interested reader. 

As the first step toward the full understanding of this new type of stochas- 
tic control problem, we shall first establish the dynamic programming prin- 
ciple and study the corresponding HJB equation. Several technical difficul- 
ties arise immediately. For example, the nonuniqueness of the solution to the 
structure equation requires a careful formulation of the control problem. For 
this reason, we formulate the control problem over the canonical Wiener- 
Poisson space. Also, since the control actions actually change the type of the 
system (as a semimartingale) , we need a general Ito formula that covers all 
possible cases in a unified form. Furthermore, it is necessary to validate the 
Bellman principle in this new setting so that all our arguments will have a 
rigorous theoretical foundation. It should also be noted that in this case the 
HJB equation takes a new form which we shall name as a "mixed second- 
order partial differential/ difference equation" depending on whether or not 
the jump part of the control is present. To our best knowledge, such a type 
of HJB equation is novel. As the main results of this paper, we prove that 
the value function is the unique viscosity solution of the HJB equation. 

The rest of the paper is organized as follows. In Section 2 we give necessary 
background theory on normal martingales and, in particular, the proper- 
ties of martingales satisfying structure equations in general Wiener-Poisson 
spaces. In Section 3 we formulate the control problem and prove the con- 
tinuity of the value function. In Section 4 we prove the Bellman principle, 
and in Section 5 we verify that the value function is a viscosity solution 
to the HJB equation. In Section 6 we prove the uniqueness of the viscosity 
solution. A counterexample that shows the nonuniqueness of the solution of 
the structure equation is given in the Appendix. 
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2. Preliminaries. Throughout this paper, we denote by (Q,J-,P;F) a 

filtered probability space, and we always assume that the filtration F = 
{Ft\t>o satisfies the "usual hypotheses" (see, e.g., Protter [20]). The fol- 
lowing notation will be used frequently in the sequel. Let (U,B(U),m) be 
a generic measure space, with the cr-finite measure m, and E be a gen- 
eral Euclidean space with Lebesgue measure. For any 1 < p < oo, we denote 
Lp([0, T] xU,dt x dm; E) to be the space of all random fields <p : [0, T] x Q x 
U i— > E such that: 

(i) for fixed uEU, </?(•, -,u) is F-progressively measurable; 

(ii) it holds that 

(2-1) E [ T Mt,;-)\\ P LP{u ^ m) dt<W. 

When p = 2 and the random fields are actually F-progressively measurable 
processes, the space is denoted by L F ([0,T];E) as usual. We shall denote 
,Mo(F,R ) to be the space of all Revalued, square integrable martingales 
X defined on (n,F,P;F) such that X = 0. 

2.1. Normal martingales and structure equations. We first recall from [8] 
that a square integrable martingale X is called "normal" if {X)t = t. Here 
(X) is the conditional quadratic variation process of X, or the compensator 
of the bracket process [X]. Since the processes [X] and (X) differ by a 
martingale, if X also has the "representation property," then it is readily 
seen that there exists an F-predictable process u such that 

(2.2) [X] t -t= t u s dX s Vt>0. 

Jo 

Equation (2.2) is called the structure equation driven by u and solved by 
X, and was first studied by Emery [10]. Examples of normal martingales 
satisfying the structure equation include the following: Brownian motion 

(u = 0), compensated Poisson process (u = a € M* = R \ {0}), in which case 
X = a(N t / a 2 — t/a 2 ), where N is a standard Poisson process, as well as 
the Azema martingale (ut = —Xt-) and the "parabolic" martingale {ut = 
-2X t -), etc. 

The general results of existence of the solution to the structure equa- 
tion have been studied by several authors, but mostly restricted to the 
one-dimensional case. For example, in Emery [10] it is proved that for any 
continuous function / :Mi->M there exists at least one X € 7Wq(F;IR) de- 
fined on a suitable filtered probability space (Q,J-,P,F), which solves the 
structure equation with m = t > 0. Moreover, in Emery [11] it is 

also shown that if / : K i— >• R is a more general deterministic function, and 
ut = f(t), t > 0, then the solution to the structure equation (2.2) exists, 
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and it is unique in law. Other references regarding the well-posedness of 
structure equations can be found, for example, in Meyer [18]; Kurtz and 
Protter [15], Azema and Rainer [5] or, more recently, Phan [19]. We should 
note that to date neither the uniqueness nor the multidimensional existence 
has been thoroughly explored. In fact, in general, even in a very special 
Wiener-Poisson space, for a given predictable process u, the solution to the 
structure equation (2.2) may not be unique, not even in law(!). We shall 
provide a counterexample in the Appendix to clarify this issue. 

To end this subsection, we list some properties of the solution of a struc- 
ture equation, which will be useful in the sequel. If A € A^q(F,R) is a 
solution to the structure equation (2.2) driven by the process u, we denote 

AX t = X t - X t -, and for each w£H, let D x (u) = {t > 0; AX t (u) ^ 0}. 
Then: 

(i) For P-a.e. wGfl, it holds that A At = ut, for all t € Djf(w); 

(ii) The continuous and the pure jump part of the martingale A, denoted 
by A c and X d , satisfy respectively, 

(2.3) dX£ = l {ut=0} dX t and dXf = l {ut ^ 0} dX u t>0. 
Finally, we recall that for any A G M 2 (F,P), it holds that 

n-l 

(2.4) [X] t = Y, (AA,) 2 + (A c ) t = hm V(A ti+1 -A ti ) 2 , 

0<s<t . =Q 

where ir : = to < • ■ • < t n = t is any partition of the interval [0, t], and | vr | = 
supj \ti + \ — ti\ denotes the mesh size of ir. Further, the limit in (2.4) is in 
probability. 

2.2. The Wiener-Poisson space. Since the well-posedness of the struc- 
ture equation is essential in a control problem, we shall first take a closer 
look at this issue, in a special probability space: the Wiener-Poisson space. 
To be more precise, let (0,JF, P) be some probability space on which is 
defined a d-dimensional standard Brownian motion B = {Bt : t > 0} and a 
(time-homogeneous) Poisson random measure /x, defined on [0,T] xl. We 
assume that B and [i are independent, and that the Levy measure of /i, 
denoted by v, satisfies the standard integrability condition 

(2.5) / (1 A \x\ 2 )u(dx) < +oo, 

JR* 

where M* = M \ {0}. For simplicity, we assume that ^({0}) = 0. We denote 
p-B,M _ {JT^'^j^Q to be the natural filtration generated by B and /j,, that is, 
Tf^ = a{B s ,/j,([0,s] x A);0 < s <t, A e B(R*)}, t>0, and denote by F the 
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augmented version of F ^ . Then F satisfies the usual hypotheses. In the 
rest of the paper we shall content ourselves to this probability space without 
further specification. 

One of the most important features for the Wiener-Poisson space defined 
above is the following martingale representation theorem (see, e.g., Jacod 
and Shiryaev [13]): for any X G M 2 (F;R d ) such that X$ = 0, there exists 
a unique pair of processes (a,(3) G L 2 ? ([0,T];R dxd ) x L|([0,T] x R*;dt x 
di>;R d ) for each T > 0, such that 

(2.6) X t = f a s dB s + f f j3 s {x)jj,{dx,ds), t>0. 

Jo Jo Jm.* 

Here jl(dtdx) = fi(dtdx) — u(dx)dt, (t,x) G [0, oo) xR*, is the compensated 
Poisson random measure. In what follows we call the pair (at,/3) in (2.6) the 
"representation kernel" of X. 

We now give a result that describes the necessary and sufficient conditions 
for a square-integrable martingale to satisfy the structure equation in a 
Wiener-Poisson space, which will play a crucial role in our future discussions. 



Proposition 2.1. Let u = {ut}t>o be a bounded, F -predictable process 
taking values in R d ; and let X G Mq(F;M. ) that has a representation kernel 
(a, (3) in the sense of (2.6). Then, X satisfies a structure equation 



(2.7) 



d[X% = dt + u\dX l t , l<i< d, 
d[X i ,Xi] t = 0, l<i<j <d,t>0, 



if and only if there are random sets A\ G £>(M*) <g) T s , s > 0, 1 < i < d, such 
that: 

(i) Efc=l a\' k ai' k = Sijl^oy, dt x dP-a.e.; 

(ii) I3\(x) = u\l A i{x), dt x dv x dP-a.e.; 

(iii) ^n^) 1 ^^^^^^ dtxdP-a.e., l<i,j<d. 
In the above, "<%" is Kroneker^s delta. 

Proof. Let X G Mo(F;R d ) be given and let (a, (3) G L|([0, T]; R dxd ) ® 
L|(([0,T] x R*;dt x du);R d ) be defined by (2.6). Then, the quadratic co- 
variation process of X 1 and X 3 , for 1 < i,j < d, is given by 

[X\Xi} t = J2 f^ k ^ k ds+ f f 0i(x)Pi(x)n(dx,ds) 
^[Jo Jo Jr* 

(2.8) = j^pi{x)pi{ x )v{dx)\ds 



,k=l 
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/o 

Here we should note that 

rt 



(3 l s (x)(3l(x)fi{dx,ds). 



E 



(2.9) 



\Pl(x)Pi(x)\fi(dx,ds) 

\(3\{x)(5i{x)\v{dx)d, 

\Pi(x)\ 2 v(dx)ds 



E 



1/2 



x i E 



\Pi(x)\ 2 is(dx)ds 



1/2 



< oo. 



Therefore, the last integral process in (2.8) is well defined, and it is a martin- 
gale obtained by compensating the stochastic integral J J R * (5 l s (x)(3 J s (x)fi(dx, ds) , 
t > 0. 

Now suppose that X satisfies (2.7). Then, plugging X with the form (2.6) 
into (2.7) and comparing it with (2.8), the uniqueness of the predictable 
semimartingale decomposition then leads us to the following identities: 

(a) u l t (io)al' J (uj) = 0, for dt x dP-a.e. (t,co); 1 < i, j < d; 

(b) Eti <xi'\u)c4> k (u) + / R , (%(x,u;M(x,u))v(dx) = S itj , for dt x dP- 
a.e. (t,x,ui), 1 <i,j < d; 



(c) ft(x,u)fi(x,u)) 



5ij ■ u l t Pl(x,io), for dt x dv x dP-a.e. (t,x,u), 1 < 



i,j < d. 

Clearly, (a) implies that '(u;)l < u i = 0, for dt x dP-a.e. (t,u>), 1 < i, 



A 



j < d. Also, if we define A\ = {(x,uj) € M* x ft : j3\{x,uj) ^ 0}, then it follows 
from (c) that 

(2.10) 0l(x,u;) = u\{u)l A i{x,u), dt x dv x dP-a.e. (t,x,u)), l<i<d, 

proving (ii). Now, setting i = j in (b) and using (2.10), we have, dt x dP- 
almost surely, 



j,fc|2-. 



(2.11) 



1 - X! \ a t I J -{uf=o} 

k=l 



+ 



u\\ 2 l-Ai( x )v{d x ) 



\a t > I 1 



K=o} 



k=l 



Multiplying l| u i =0 | and l^ u i^ on both sides above, respectively, one can 



easily see that J2t=i \ a t k \ 2 



1 on {u\ = 0} and u(A*) = ^ on {uj + 0}, 
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1 < i < d, dt x dP-a.e. On the other hand, setting i ^ j in (b) we have, for 
dt x dP-a.e., 

= f:«i' fc «f + / (il{x)(3l{x)v{dx) 

(2.12) 

d 

= E «t'V fcl K=o^=o} + u l u t"( A l n M)- 

k=l * 

We can then easily check that v{A\ n >r|) = on the set ^ 0,u J t ^ 0}, 

hence, (iii) holds. This then further implies that J2k=i °h! k o4' k = 0, dt x dP- 
a.e., proving (i). 

Conversely, if X G Ml(F;R d ) has the form 

X t = X + [ a s dB s + f f p s fl{dx,ds), t>0, 
Jo Jo Jr* 

with (a,P) satisfying (i)-(iii), then a straightforward calculation starting 
from (2.8) shows that it satisfies the structure equation (2.7). The proof is 
now complete. □ 

As a direct consequence of Proposition 2.1, we can now prove an existence 
result for the structure equation on the Wiener-Poisson space. We note that 
a Levy measure can have at most countably many atoms, we shall focus on 
the continuous part of the Levy measure. In other words, denoting the set 
of all atoms of v by T, thus, the "continuous part" of u, denoted by u c , is 
defined by 

v c {A) = v{A)- £ u({z}) VAgB(R*),0^, 

where A is the closure of A in M. We have the following existence result for 
a (possibly) multidimensional structure equation. 

Theorem 2.2. Assume that v c {[— 1, 1]) = +oo. Then, for any bounded, 
¥ -predictable process u t = {u\ , . . . , uf), t>0, the structure equation (2.7) 
has at least one solution in the Wiener-Poisson space (Q, J 7 , P, F,B,fx). 

Proof. For each t > 0, let us define random times 1 = t f > t\ > ■ ■ ■ > 
Tf > inductively as 

sup{r < tT 1 : v c ({-rt\ -r] U [r, r^)) = K)" 2 }, K + 0, 
r\~\ u\ = 0. 

This is always possible because z^ c ([— 1,1]) = +oo. Clearly, all r"s are T%- 
measurable. Moreover, the mapping (t,uS) \— * t\{lS) is jointly measurable on 



(2.13) r\ = { 
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M* x Q. In light of Proposition 2.1 we see that, for a given bounded, F- 
predictable process u = (it 1 , . . . ,u d ), we can construct a solution X to the 
structure equation (2.7) with the form of (2.6) by simply setting 

A = K-rt'-rH U [rlrl 1 )] n T c , 1 < i < d, 

(2-14) 

l<z,j<d,xeR*,t>0. 

It is easy to check that the conditions (i)-(iii) in Proposition 2.1 are satisfied, 
hence, the conclusion follows. □ 

To conclude this section, we give an ltd formula for processes that satisfy 
an equation of type (2.7). 

Proposition 2.3. Let u = {ut,t > 0} be a bounded F -predictable process 
with values in M. d and X G .Mq(F; M d ) a solution to the associated structure 
equation (2.7). Then, for any function if G C 1,2 ([0, T] x M. d ), the following 
formula holds: 

cp(t, X t ) - <p(0, 0)=]T f < [if] (s, X s _) dxi 

(2.15) 

+ f X s) + %u s [<p]{s, X s )^j ds, 

for all t € [0, T\, with 

sir -u \ t 9 . . , „ <p(s,x + u % ei) - <p(s,x) 
w u Yp\{s,x) = l {ul=0} —tp{s,x) + l {n¥0 } ~i . 

d ( Id 2 
(2.16) ^ u [ip](s,x) = Y^[ 1 ^=o}^Q^2 l P(s,x) 

ip{s, x + u l ej) - ip(s, x) - u^ipjs, x) \ 
+ 1 {«¥o} 7yj2 J > 

where {e\, . . . ,ej} is the canonical orthonormal basis in M. d . 

Proof. We first apply the general ltd formula to get, for all t £ [0,T], 
ip(t,X t )-ip(0,X o ) 

= J^^s,X s) ds + ±J^,(s,X^dXl 

(2.17) 
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+ Yl (<p{s,X,)-v(s,X,-)-Y J -§-<p(s,X,-)&X i \ 

0<s<t \ i=l ' / 

Since AX l AX J = A[X l ,X J ] = 0, whenever i ^ j, we see that at any given 
time at most one coordinate of X can jump. Furthermore, note that the jump 
size of the process X 1 is determined by AX* = u l s , whenever AX l s ^ 0. Thus, 
by a standard calculation using (2.7) and relation (2.4) for each X 1 , 1 < i < d, 
we can rewrite the last sum on the right-hand side of (2.17) as 

A ^ <p(s,X s ^ +u i s e l ) - <p{s,X s „) - £-<p{s,X s _)ui 



i=10<s<t \ «' 

_A f\ ip(s,X s _+uiei)-tp(s,X s _) - ^-cp(s,X s _)ui 



i=i 



X (l{n^0}^ + <^)- 

Finally, note that [X\ X j ] = 0, for all 1 < i < j < d. We see that ((A^) c , (X?') c ) 
0, for i^ 1 j, and the result follows easily. □ 



3. The stochastic control problem. In this section we formulate our con- 
trol problem and give some preliminary results. Due to the technical sub- 
tleties involved in the solutions of the structure equation as we indicated in 
Section 2, we find it more convenient to work on a canonical Wiener-Poisson 
space, which we now describe. 

Let T > be an arbitrarily given finite time horizon. For any < s < t < 
T, we denote Ql t to be the space of all continuous functions from [s,t] to 

R" starting from 0, endowed with the sup-norm. We define B? = S8{Sl\ t ) to 
be the topological cr-algebra of ft] t , and let P} t be the Wiener measure on 

Next, we denote £l 2 st to be the set of all N-valued measures q on ([s,t] x 
R*,^([s,i] x R*)) and denote B\ to be the smallest cr-algebra to which 
all mappings q G fif t i-> q(A) GZ + U {oo}, A G 3S([s,t] x R*), are measur- 
able. Let Hs,t{q, ■) = <Z0 G t to be the coordinate random measure defined 
on (r2g t , £?!>), and denote P 2 t to be the probability under which /i S]t is a 
Poisson random measure with Levy measure v satisfying (2.5) and that 
y c {[— 1,1]) = +oo. We note that the second condition is merely technical, 
which will guarantee the existence of solutions to the structure equation, 
thanks to Theorem 2.2. 



STOCHASTIC CONTROL FOR NORMAL MARTINGALES 11 
We now define, for each < s < t < T, n Sjt = nl t xfl 2 st ; P s>t = P} t <g> P 2 t ; 



and B s j = B\ ® &\ , the completion of ® B\ with respect to the prob- 
ability measure P S) t- The canonical Wiener-Poisson space is then defined 

as (ri,jF, P) = (f2o,T ; So,T,-fo,T)- We shall denote the generic element of 

(f2, T, P) by uj = (uji, W2), where lo{ 6 ff 0T , i = 1, 2; and we define 

B t (u)=u 1 (t), fi(uj,A)=uj 2 ( y A) 

Vte[0,T\,weSl, iGf([0,T] xR*). 

Clearly, under the probability P, the process B is a standard Brownian 
motion, and [i is a Poisson random measure with Levy measure and they 
are independent. 

Finally, for t € [0, T], we shall denote F* = (^) sg [o,T] to be the filtration 
on [t, T] in the following sense: 

Fl = a{B r -B t ,n(A):A<E^(lt,r] xR*),t<r<s}, t<s<T, 
and T\ is a trivial c-field for all s <t. We assume that all F*'s are aug- 
mented by P-null sets. In particular, we set F = F°, and JF S = JTj?, for all 



s 6 [0, T]. Furthermore, we denote by M 2 (F;R d ) the set of all Revalued, 
square-integrable F-martingales X on (Q,J-,P;F). 

We are now ready to describe our control problem. Let U\ and U d be two 
nonempty compact sets in R and R d , respectively, and set U = U\ x U d . 

Definition 3.1. For all t £ [0,T], we say that a triple (ir,u,X) is a 
"control at time t" if the following properties hold: 

(i) (vr,n) is a pair of F'-predictable processes with values in U; 

(ii) XeM 2 (F;R d ) satisfies 



(3.1) 



[X i ] t = t + J uidxi, l<i<d,te[0,T], 

[X\Xi] t = 0, l<i<j<d,te[0,T], 
X s -X t l±T t , Vs>t. 



Here "_I_L" stands for "independent of." We denote by U(t) the set of all 
controls at time t. 

The following proposition gives two basic properties o£U(t). 

Proposition 3.2. (i) For any t e [0,T] and any U -valued F t -valued 
predictable couple of processes (vr,u), there exists X G A / ( 2 (F;R d ) smc/i i/iai 
(tt,u,X) eW(i); 

(ii) For any 0<t<t' <T, it holds that U{t') C U{t). 
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X x 



Proof. The proof of (i) is quite straightforward. For any t G [0,T], we 
first apply Theorem 2.2 to obtain a solution to the structure equation (2.7) 
on [0,t], denote it by l X. We then apply Theorem 2.2 again to find a square 
integrable, F'-martingale {X^s G that solves the structure equation 

on the interval [t,T]. We now define a process 

se[0,t), 

X*-X* + *X tj s6[t,T]. 

One can easily check that X satisfies (3.1). 

To see (ii), first note that, for all s > 0, T\ C T\. Thus, if (vr,u) is 
F* -predictable, it is also F*-predictable. Now, let u be a Z7 d -valued F* - 
predictable process and X be the corresponding solution of (3.1). Since F* 
is trivial before t' , u must be a.s. deterministic, and equation (3.1) restricted 
to [0, t'] has a solution X of the form 

rs rs r 

K= 1r=o}^+ / <U^)Kdx,ds), se[0,t'], 
Jo Jo Jr* 

where A l r is defined as the one in the proof of Theorem 2.2. In fact, this 
solution is unique in law, since u is deterministic on [0, t']. It then follows that 
the process {X s ,s G [0, t']} is of independent increments. Hence, X s — X t _LL 
Tt , for s G [t, t'] . In particular, we have X# — Xt _LL Tt- Finally, for s G (t', T] , 
we write 

X a -X t = {X s -X t ,) + {X t ,-X t ). 

Since X s — X t > is independent of Ty by definition, whence of Tt-, we conclude 
that X s — Xt is independent of J~t, proving the proposition. □ 

We now describe the main ingredients of our control problem. Given any 
initial data (t, y) G [0, T] x W 71 and a control a = (tt, u, X) G U(t), we assume 
that the controlled dynamics Y t,v,a = Y satisfy the following SDE driven by 
the normal martingale X: 

rs rs 

(3.2) Y s = y + J b(Y r , 7r r , u r ) dr + J a(Y r -,ir r ,u r ) dX r , sG[t,T], 

and we make the convention that Y s = y, for all s < t. We consider the 
following cost functional for the control problem: 

(3.3) J(t,y;a) = E[g(Y^ a )}, (t, y) G [0, T] x M m , 
and therefore, the value function is given by 

(3.4) V(t,y)= inf E[g(Y^ a )}, (t, y) G [0, T] x M. m . 

a£U(t) 

Throughout this paper we shall make use of the following assumptions on 
the coefficients b, a and g. 
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(HI) The functions b : M. m x U -> R m and a : R m x U -> M mxd are uniformly 
continuous in (y, 7r, u) and Lipschitz in y, uniformly with respect to 
(ir,u). 

(H2) The function g : W 71 — ^ 1R is bounded and continuous. 

It is well known that, under the assumptions (HI) and (H2), for any 
given control a £ W(0), there exists a unique F-adapted continuous solu- 
tion to (3.2), that we denote by Y*- y ' a = {Y^> a ) s > t , or for simplicity. 
If a EU(t), then Y t,y is F*-adapted. A simple application of Ito's formula 
(Proposition 2.3) yields the following result regarding the second-order op- 
erator associated with the process Y l ' v . We state it as a ready reference, but 
omit the proof. 



Proposition 3.3. Let a = (tt,u,X) gW(0) andY =Y f ' y be the unique 
solution of equation (3.2). Then for any ip £ C 1,2 ([0, T] x M m ) ; it holds that 

ip(s,Y s ) - <p(t,y) 

(3-5) = £ [ S K rtU M(r,Y r -)dXi 

i=i Jt 

+ J* (j^<p(r,Y r ) + JS$ rpj11p [ ¥ >](r,Y r )) dr, s G [i,T], 

where 

K,uM(r,y) 

= 1 {K=0}^ yf(r, V)o l (y, vr, u) 

<p(r, y + u l a l (y, tt, u)) - (f(r, y) 
+ i {«¥0} ^ > 

•^r,«M(^y) 

= Vy<p(r,y)b(y,ir,u) 

d 

+ E( 1 {« 1 =0} 3 (Dyvfi 3 ' ^KC^ 7r ' u )> ^ G/> 7r ' M )) 

i=l 

+ i{«^o}(v( r ' y + uV (y> tt, «)) - 9?( s , y) 

-u*V y ip(r,y)o-\y,7r,u))/(u*) 2 ). 
Above, a 1 denotes the ith column of the matrix a. □ 



We now give a useful result of the value function. 
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Proposition 3.4. Assume (HI) and (H2). Then, the mapping (t,y) ^ 
V(t,y) is continuous on [0,T] x M m . 

Proof. It suffices to show that, for any M > 1, V(-, •) is continuous on 
[0,T] x Bm(0), where Bm(0) denotes the closed ball in M. m centered at 0, 
with radius M. 

To this end, let < t < t' < T and y,y' £ B M (0). Recall from Proposi- 
tion 3.2(h) that U{t') cU(t). One can deduce by standard arguments that, 
for some C > 0, depending only on the coefficients b, a and g, the following 
inequalities hold for all < t < t! < T and y, y' € M m : 

sup e{ sup \Yt y \ 2 T t X<C(l + \y\ 2 ), 

a€U(t) Ue[t,T] J 

(3.6) sup e! sup ly^-y/^'i 2 

oew(t) Le[t',T] J 

^^(l + lyp + ly'l^lt-t'l + b-y'l 2 ), 

where K** and Y* ,?/ are the solutions of (3.2) starting from (t, y) and (t', y'), 
respectively. 

Now, for any e > 0, there is some a = (ir, u, X) £ W(t) depending on (i, y), 
such that 

(3.7) F(i,y)>Eb(l^)]- e . 

Also, applying (3.6), we can find, for the given e > 0, some N = N £ > 1 such 
that 



(3.8) sup P{\Y£ y \V\Y^ y '\>N}< ~"\' T2 ^ ' <- 

aeU(t) 



2C(1 + M 2 ) e 
iV2 " 2' 



Since y is bounded and continuous, thanks to (H2), for the given e > 
and N = N £ > , there exists <5 = S(s,N e ) > such that |y(z) - g(z')\ < e, 
whenever z, z' € -Bat(O) and |z — z'\ < 5. Consequently, we have 



E{\g(Y^) - g(Y^y )\} 
(3.9) < e||^||oo + - fl'(^* V ')|l < | 1 ^.»,< J v,| V ^.v'|< JV> } 



< e(l + Halloo) + 2||y|| 00 P{|Y^ - Y^ y | > 5}. 
Combining (3.9) and (3.6), we obtain that 

i?{| 5 (y'^ y )- 5 (y'/^ , )|} < e (i+|| 5 || 0O ) + ^|| 5 || 0O c'{(i+2iv^ 2 )|t-tn + |y-2/r}- 
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Since V(t',y') < E{g(Y^' v )} always holds, it follows that 

V(t,y)-V(t',y') 
(3.10) >E{g{Y^)-g{Y^')}-e 

> -e(\\g\\oc + 2) - JhI<?HocC(1 + 2M 2 ){\t - t'\ + \y- yf}. 



Taking liminf over (t,y), (t' ,y') G [0,T] x Bm(0), we obtain that 

(3.11) liminf {V(t, y) - V(t', y')} > 0. 

It is clear that to prove the continuity of V it suffices to prove the following 
inequality: 

(3.12) hmsup (V(t,y)-V(t',y'))<0. 

0<t'-t\0,|2/-y'|-»0, 

We again follow a more or less standard procedure. Namely, we first choose 
a' = (tt', u', X') G U{t') so that 

V{t',y')>E[g{Y^')]-e. 

Then, following the same estimates as was done for (3.10), we can derive 
that 

V(t,y)-V(t',y') 

(3.13) <E[g{Y^)]-E[g{Y^')]+e 

< £(|b||oo + 2) + 2||< 7 || 00 cl((l + 2M 2 )\t - t'\ + \y - y'\ 2 ). 



Taking limsup for (t,y), (t',y') G [0,T] x B M (0), we derive (3.12). 

Combining (3.11) and (3.12), we see that, on [0,T] x B M (0) the limit 
exists and 

(3.14) lim (V(t,y)-V(t',y')) = 0. 

It then follows that 

lim \V(t,y)-V(t',y')\ 
\v-t\+\v-tf\-> > 

(3.15) 



lim (V(s,z)-V(s',z')) 

0<s'-s\0,|2-z'|^0 



0, 



where s = t A t', s' = t V t', (z, z') = (y, y') if t < t' and (z, z') = (y\ y) other- 
wise. Consequently, V is continuous on [0, T] x Bm(0), and the proposition 
follows. □ 
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4. The Bellman principle. In this section we prove the following 11 dy- 
namic programming principle" (Bellman principle) for our control problem 
(3.2)-(3.4). 

Proposition 4.1 (Dynamic programming principle). Assume (HI) 
and (H2). Then, for any (t,y) G [0,T] x W l and 0<h<T - t, it holds 
that 

(4.1) V(t,y)= inf E[V(t + h,Yffi)). 

a&t(t) 

PROOF. We fix < t < t + h < T, y G R m and a = (tt,u,X) G U(t). We 
denote the corresponding solution to equation (3.2) with initial data (t,y) 
and control a by Y t,y . For each n > 1, we set T n = {k2~ n ; k G Z m }, and for 
each z = k2~ n G T n , we consider the m-dimensional cube I(z) := 111=1 [(^-i — 
1)2-^2-™). 

Obviously, for any e > 0, n > 1, and z G T n , we can find some control 



z = (tt z ,u z ,X z ) G W(t + h) such that the associated solution Y z = Y t+h > z > a * 



satisfies 

(4.2) E[g(YS)] = E[g{Y^ aZ )\ < V(t + h,z)+ e. 
Then we define the new control pair: 

(ao\ (~ ^^_[{^s,u s ), if sG (t,t + h), 

(4.3) {tt s ,u s ) - I (7rf ^ f)) . f y £„ e ^ se[t + Kn 

and the process 

(AA\ Y ifsG[0,t + /t), 

1 j As "lli +fc + (X|-J^), ify/^G/(z) andsGft + ^T], 

where z runs over all T n . It is not hard to check that the process X is 
a solution of the structure equation (2.7) driven by u; and (tt,u,X) is a 
control in U(t). Let us denote by Y the corresponding solution to equation 
(3.2) with initial data (i, y). Since the processes {(7r s ,u s , X s ), s G [t,t + fa]} 
and {(ir z ,u z , X z — X* +h ), s G [i + h,T],z G r n }, are independent, so are the 

solutions {(Y s ),s G + and {(7/),s€ (t + /i,T]}, z G F 1 . Furthermore, 
the uniqueness of the solution to SDE (3.2) implies that Y s = Yg' y , for all 
s G [t,t + h]; and for s G [t + h,T], P-a.s. on the set {Y*f h G /(z)} it holds 
that 

Y s - Y* = (Y t % y h -z)+ [ S (b(Y r , n r ,u r )- b(Y r z ,7t r , u T ))dr 

Jt+h 



t+h 



H+h 

(cr(Y r ,Tr r ,u r ) — a(Y r z ,7r r ,u r )) dX r . 
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Thus, with some standard estimates and using the fact that X is a normal 
martingale, as well as Gronwall's inequality, we obtain that there is some 
constant C > (depending only on the Lipschitz constants of the coefficients 
b and a), such that 

(4.5) e{ sup £ \Y s -Y?\H I{z) (Y^ h )\<CmA-\ 

[se[t+h,T] zeTn J 

Consequently, from (3.6) and the continuity and boundedness of the func- 
tion g, we can deduce that, for some n £ > 1 and all n>n e , 



(4.6) 



e{ £ \9(Yt) - g(Yf)\l I{z) (Y t %)\ <e. 
Uer» J 



Moreover, since the Y^'s and Y t i_ y h are independent, for every n > N(e), we 
have 

E{g{Y T )}<E\ J2 g(YS)l Hz) (Y t %l)\+e 

= J2E{g(YS)}P{Y t %eI(z)} + s 

z£F n 

< £ (V(t + h,z) + s)P{Y t % e I(z)} + e 
zer™ 

= V(t + h,z)P{Y t %eI(z)} + 2s 

z£T n 

= e\ J2 V{t + h,z)l I{z) (Y^ h )\+2e. 
lzer n ) 

Letting n — > +oo and applying Proposition 3.4, we obtain from the bounded 
convergence theorem that 

V(t, y) < E{g(Yr)} < E{V(t + h, Y^ h )} + 2s. 

Finally, since a = (tt,u,X) & U(t) is arbitrary, we conclude that 

V(t,x)< inf E{V(t + h,Y t %Y)}. 

To prove the converse inequality, we borrow some idea of [12]. Namely, we 
shall split the canonical probability space into two, and patch up the desired 
control. 

First recall that each canonical space f2 Sj j is the product of the canoni- 
cal Wiener space and the canonical space of the Poisson random measure, 
thus, we can identify the probability spaces (f2,F,P) = (£\>,T> Fq,t 5 Pot) 
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with (ft 0) t+/i x ^t+/i,T,F , t +h <g> F t+htT ,P j +h ®P t+h)T ) by the following bi- 
jection p : Oo,t i— > ^o,t+h x ^Vf-h,T- For w = (^1^2) S ^o,T = ^0 T x ^0 T' we 
define 

A 

UQ,t+h = (^l\[0,t+h]^2\[0,t+h]) G ^0,t+h, 
' ^t+h,T = {{u)\- Ui(t + M)l[t+/i,T],^2|[t+/i,T]) G Qt+h,T, 

p(u) = (u 0>t+h ,uj t+htT ). 

Now, for e > 0, let a = (it, u, X) € U{t) such that 

^(y^' a )]<y(t,y) + e. 

For all woj+h £ ^o,t+/i) we define a process on Q t +h,T by 

(7r(6J 0it+h ,-),n(w 0it+h ,-))(w t+/liT ) = (vr,u) op~ l {^t+h,^t+h,T)- 

For P ,t+ft-almost all ^o,t+h, (^{^o,t+h,-),u(u;o,t+h,-)) is a version of an 
Ft+zj ^-predictable process and, following the construction of Proposition 
2.1, we can prove that, if X solves the structure equation (2.7) driven by u on 
[0,T], then X(u 0}t+h ,-) = X o p- 1 (u 0jt +h, solves (2.7) driven by u(u} 0jt+h ,-) 
on the time interval [t + h,T]. 

We now consider the control process a(ujQ t t+hr) = (i"(^o,t+ft) •)> ^0^o,t+ft) 
•),X(w ,t+/ir))- For each w , t +h G ^o.t+fc, we denote £(w ,t+fc) = 
^+ft°[P~ 1 (' t; o,t+/ii ")] e We note that Y^J^ a is JF i+fc -measurable, 

thus, £ depends only on ct>o,t+/i- Next, we denote the solution of (3.2) on 

[t + h,T] x n t+htT with initial condition £(uj ,t+h) by y t+h ' 5 ' ct (u;o,t+/ l )(-) = 
Yt+h4(u ,t+h)Xuo,t+h,-) . Then, as in [12], one shows that for P 0)t+/l -almost 
all (Vo,t+h G ^o,t+/i> the paths of y*' 3/ ' a [p" 1 (wo,t+/i, •)] coincide with those of 
Y t+h, £ ,a (uoj + h)(-), Pt+zj^-almost surely. Moreover, for P ,i+fe-almost every 
uoj+h, it holds that 

E{g(Y^ a )\^ t+h }(p-Hu , t+h ,-)) 

- b ,t+h{g{y T )}l 2 =^;o P -i(, 0it+h ,)- 

Pt+h^-almost surely. Now, for any given ttq £ U\ , we put 

{7f(^0,i+ft, •) = 7Tol[o,t+/i) + 1T(uJ 0t t+h, -)l[t+h,T], 
u(^>0,t+h, •) = U(u> f+h, -)l[t+h,T] ) 
-X" =- B l[0,t+h) + (^(^0,t+h, •) - ^+/i( w 0,t+ho •) +- B t)l[t+fe,T]- 

Then, clearly a(wo,t+/i , •) = (vf(w ,t+h, -),u{^ t +h, ■),'X U) °' t+h ) eU{t + h). Con- 
sequently, by definition of V(t + h,z), we have for all z € R m and Po,t+h-a.s. 
every w ,t+h, 

^(y^'^ ^' )] = ^(y^'* ^' )] > v(t + z). 
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This implies that 

V{t,y) + e>E[E[g{Y^ a )\F\ +h \] 
(4.7) > E[V(t + h, z) z=Y t, y , a ] = E[V{t + h, Y^ a )} 

> inf E[V(t + h,Y*' y ' a )}. 
aeu(t) 

Since this is true for all e > 0, we get the second inequality. This completes 
the proof. □ 

5. The HJB equation. In this section we apply the Bellman principle of 
the previous section to derive the corresponding Hamilton-Jacobi-Bellman 
(HJB) equation for our control problem. To be more precise, we shall prove 
that the value function is a viscosity solution of the following fully nonlinear 
partial differential-difference equation (PDDE): 

^-n \-^ v ^y)~ ^_^,u[V](t,y) = 0, (t,y)e[0,T]xR m , 

(5.1) < at (ir,u)eu 

{v(T,y)=g(y), y € M m , 

where the second-order operator J^ )U is of the form (3.6), 

= V y ip(t,y)b(y,-K,u) 

d 

(5.2) +£{1 

{u*=0} 2 

{D 2 yy ip{t, y)a\y, vr, u) , a 1 (y, vr, u)) 

i=i 

+ !{wVo} (v^i v + ul(J% {y-> 7r ^ u ))- wit, y) 

- u i V y ip{t,y)a i {y,TT,u))/(u i ) 2 }, 

(5.3) tp€C 1 > 2 ([0,T\ xl m ). 

Here again, o % denotes the ith column of the matrix a. We shall refer to 
equation (5.1) as the HJB equation in the sequel. We should note that such 
a second-order PDDE has not been studied systematically in the literature, 
therefore, in what follows we give some detailed investigation regarding the 
viscosity solution to such an equation. We begin by introducing the notion of 
viscosity solution, following the approach by Barles, Buckdahn and Pardoux 
in [6] (see also Alvarez and Tourin [1] and Amodori, Karlsen and La Chioma 
[2])- 

Definition 5.1. A continuous function V : [0,T] x R m — > R is called a 
viscosity subsolution (resp. supersolution) of the PDDE (5.1) if: 
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(i) V(T,y) < (resp. >) g{y), y € M. m ; and 

(ii) for any (t, y) e [0, T) x R m and if € C 1 ' 2 ([0, T] x M m ) such that V - <p 
attains a local maximum (resp. minimum) at (t,y), it holds that 

(5-4) - %-<p(t,y) - M_^\VM(t,V) < (resp. >) o, 

for all sufficiently small 5 > 0, where 

< u rM(*,y) 

d 
i=l 

+ !{0<|^|<5}(^(^y + wV(y,vr,u)) - <p(<,y) 

(5.5) (y(t,y + «V(y,7r,«))-F(t,y) 

(5.6) - u 4 V^(i, y)a i {y, tt, u))/^) 2 }, 

(p€C 1 ' 2 ([0,T\ xR m ). 

A function V is called a viscosity solution of (5.1) if it is both a viscosity 
subsolution and a supersolution of (5.1). 

Remark 5.1. We note that the last two second-order difference quo- 
tients in (5.5) are designed to take away the possible singularity at u = 
when V is not smooth. Such an idea was also used in [6]. 

In the general theory of viscosity solutions one can often replace the local 
maximum and/or minimum in the definition above by the global ones. We 
shall verify that this can be done in our case as well. The proof follows 
largely the idea of [6], Lemma 3.3. 

Lemma 5.1. In Definition 5.1 one can consider only those test func- 
tions (p £ C 1,2 {[0,T] x R m ) such that V — (p achieves a global maximum {for 
a viscosity subsolution) and a global minimum {for a viscosity supersolu- 
tion), respectively, at {t,y). Furthermore, the operator Jk?£ u [V,tp]{t,y) can 
be replaced by Jj? niU [p]{t,y) defined by (5.2). 

Proof. We shall prove only the supersolution case. The proof of the 
subsolution case is similar, but easier. We leave it to the interested reader. 
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Let us first assume that V is a supersolution of (5.1) in the sense of 
Definition 5.1. Fix any (t, y) G [0, T) x M m , and assume that ip G C^QO, T] x 
M m ) is such that V — <p attains its global minimum at (t, y) [i.e., V(s,z) — 
<p{s,z)> V(t, y) - tp(t, y) for all (s, z) G [0, T] x R m ] . Then, for all (vr, u) G U, 
and i = 1, . . . , m, it holds that 

V(t, y + uV(y, ir, u)) - V(t, y) > <p(t, y + uV(y, vr, u)) - <p(t, y). 

Plugging this into (5.5), we obtain 

J2£ u [V, (t, j/) > JZ«, U [<p] (t,y), (tt, u) G U. 

Moreover, since is a supersolution in the sense of Definition 5.1, we have 

-^T^(^y)- inf _^r,uM(*,y) 

(7r,it)eC/ 

(5 - 7) 

>- T Mt,y)~ M_S?l u [V,ip}(t,y)>0, 

for all 5 > 0. Namely, (5.4) holds when U [V, is replaced by C^^ u {f\. 

We now prove the converse. Let (t,y) G [0, T) x R m be fixed, and let 
99 G C^QO.r] x R m ) be such that V(s,z) - (p(s,z) > V(t,y) - tp(t,y) for 
all (s,z) in some (^-neighborhood of (t,y) in [0, T] x M. m . We define a new 
function 

i/>(s,z) = <p(s,z) - (f(t,y) - V(t,y)) - p\(s,z) - (t,y)\ A , 

(s,z) G [0,7] x R m , 

for some p > sufficiently small. By changing this function outside the So/2- 
neighborhood of (t,y) if necessary, we can assume without loss of generality 
that tp(t,y) = V(t,y), and 

V(s,z) > 4,{s,z)+p\{s,z) - (t,y)\ 4 V(a,z) G [0,T] x M m . 

Noting that (JL, V y ,D% y )^(t,y) = (^,V y ,D^ y )(p(t,y), we have 

|^f iM [y,^](t,y)-^jy^](t, y )| 



(5.8) 



< S 1 {0<kl<5}ZT[2 K^(*»f + - ^(*» f)) 

t=l v u J 

-(^(t,y + nV)-(^(t,y))| 

<C2(l + |l/|) 4 d*V 



whenever <5 < 2C fe^n ; where C a > is such that |c| = \cr(y, ir,u)\ < C a (l + 
\y\), (vr,u) G U. 
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Next, let x € C°°([0, T] x M. m ) be a nonnegative function with supp(x) C 
{(s,z) G [0,7] x M m :\(s,z) - (T/2,0)\ < 1}. Since V is continuous, for any 
£ > 1, we can find ^ G (0, such that, for all (s,z), (s',2 7 ) G [0,T] x R m 
with 

- (t,y)\ V |(aV) - (t,y)\<e + l, \(s,z) - (s',z')\ < u e , 
it holds that 

1 



\(V -^){s,z)-{V -^){s\z')\<p- {2i 
We now set Xe{s,z) = ^+rX(j- e (s, z)), (s, z) G [0,T] x R m , and define 

Ms, z) = Ms, z)+ f (V{s\ z>) - iP(s>, z') - p-±-) 

( 5 - 9 ) x 1 W,z>)-{t,y)\e[±,l+i]} 

x Xe{{s,z) - (s\z'))ds dz' . 

It can be easily verified that the sequence {ipe}e>i C C°°([0,T] x M m ) enjoys 
the following properties: 

(i) Ms,z) =i>(s,z), for all (s,z) G [0,T] x R m with \(s,z) - (t,y)\ <£ 

(3^ + 2); 

(ii) l/(s, z) > ^(s, z) > if>(s, z), for all (s, z) G [0, T] x K m with | < |(s, z) - 
(t,y)\<£; 

(iii) Ms,z) > V(s,z) - p£~ 4 , for all (s,z) G [0,T] x R m with |(s,z) - 

We observe that these properties imply, in particular, the uniform con- 
vergence of tpi to V on all compacts in [0,T] x M m . Moreover, due to our 
assumption, the fact that V — tpi achieves a global minimum at (i,y) implies 
that, for all £> 1, 

— 57<M*,2/) - mf _^t,uipe{t,y) > 0. 

Ct (vr,n)G!7 

Hence, we can find for every £>1 a control state (tt£,U£) G £/ such that 

(5.10) - J^(t, y) - JS^^t, y) > -1/1 

By extracting a subsequence if necessary, we can assume without loss of gen- 
erality that the sequence {(ire, ue)}g>i converges to some (tt, u) G f7 (recall 
that U is compact). Now, 

d 

(5.11) jSf T<}U< y) = V y Mt, y)Ky, Tv e , ut) + i 

1=1 
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where 



A 



+ 1 {o<\ u \\<5}ti>i{ t iy+ u \p l ty^i,ui)) - i/>(t,y) 

(5.12) - u\V y iP(t, y)a l (y, n e , u e ))/(4) 2 

+ 1 {\u\\>5}{^ii t ^y + u \ a% (y^ u d) - ^(t,y) 

- v} £ V y i){t, y)a\y, ir t , u e ))/(u}) 2 . 
Now, since ipi converges to V uniformly on compacts, it follows that 
( V(t, y + uV%, 7T, u)) - V(t, y) - uW y ^(t, y)a t {y, vr, u) 



(«*) 



lim Q\ = ( 

£— >oo 



\u l \ >0; 



if there is an infinite subsequence of u\ with u\ = Q. 

Note that the term in (5.12) that involves l{o<M|<5} does not necessarily 
have a limit, but since ipe > ip, we can at least conclude that, for all 1 < i < d, 

liminf 6| > l {ui=0} l(D 2 Jj(t,y)a l (y,Tr,u),a l (y,TT,u)) 

i— >+oo L ' aa 

+ 1 {o<i^i<<j}(^(* ) y + ^ cr *(y> 71 ")^)) - 4>(t,y) 

(5.13) -tfVyr/tfay^yw))/^) 2 

+ 1 {\ui\>8}(V(t, y + uV (y, 7T, u)) - V(t, y) 

-u i V y ^t,y)a i (y,7r,u))/(u i ) 2 . 

Thus, in light of (5.11) we obtain that liminf£_+ +00 J^ iUj ,^(i,y) > 
^^ u [V,tp](t,y). Combining with (5.10) and (5.8), we have 

o<-^(t, y )-j?l u [vM(t,y) 

< ~<p(t,y) - Sf° iU \V,<p](t,v) + Ct(l + \y\) A dp5 2 , 

whenever 5 < 2C n+ra] • Therefore, first letting p — > and then taking the 

infimum over all control states (tt, u) 6 U, we see that V is a viscosity su- 
persolution. □ 



Our main result of this section is the following: 
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Theorem 5.2. The value function V(t,y) defined by (3.4) is a viscosity 
solution of (5.1). 

Proof. We first show that V(-,-) is a subsolution. Let (t,y) G [0,T) x 
IR m be fixed. Given an arbitrary deterministic constant control (n, u) G U, we 
consider a= (n,u,X) El4(t) and denote by Y t,y the corresponding solution 
to SDE (3.2), as usual. From Proposition 4.1 we see that, for all h > with 
t + h<T, it holds that 

(5.14) V(t,y)<E»{V(t + h,Y t %l)}. 

Now let ip G C 1,2 ([0, T] x R m ) be such tat V — ip achieves a global maximum 
at (t,y). Then, by Ito's formula, one has 

< E*{V{t + h, Yg h ) - V(t, y)} < E»{<p(t + h, Y^ h ) - <p(t, y)} 

rt+h 

= J E^{d s ip{s,Yt V )+^uW^Yt v )}ds. 

Since the process Y t,y has right-continuous trajectories, we see that the 
mapping s h- ► E^{d s ip(s,Yg ,y ) + «5f T]U [</7](s,y^ ,y )} is right-continuous. Hence, 
dividing both sides of the above inequality by h and taking the limit as h 
tends to zero, we obtain that 

-J^(t,y)-X,u^(i,y)<0 V(vr,u)eZ7. 

It then follows from Lemma 5.1 that V is a viscosity subsolution. 

We now prove that V(-, •) is a supersolution. Again, let (t, y) E [0, T) x M m 
and if G C 1,2 ([0,T) x W 71 ) be such that V — ip attains a global minimum at 
(t,y). Since the value of ^Cn,u < p(t,y) depends only on the values of ip in a 
C(l + |y|)-neighborhood of (t,y), which does not depend on (ir,u) G U, we 
can change the values of ip outside of this neighborhood without changing 
inf 7r>u «Sf 7r)U </?(t,y). Further, since V is bounded, we can also assume without 
loss of generality that the functions <p, as well as all its derivatives are 
bounded and such that, for some constant C, \ip(s, z)\ < C(l + l^j) -2 , (s, z) G 
[0, T] x W 71 , with ip = d s ip, V y p, Dy y p, respectively. In particular, we see that, 
for some C > 0, 

\Sf v M{s,z)\<C, (s } z)e[0,T) xR m , (tt,u)gU. 

On the other hand, from the continuity of the mapping (s, z, tt, u) — > J^(p(s, z) + 
■^-K ,u\<p\(s , z), it follows that for an arbitrarily given e > there is some 5 > 
such that 



(5.15) \(d s ip(s, z) + J2f w ,uM(a, z)) - (d 8 ip(t, y) + ^, u [p](t, y))\<e, 
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for all (s,z) G [0,T] x M. m with \(s,z) - (t,y)\ <25, and for all (tt,u) G U. 

Let h G (0, 5). By the Bellman principle (Proposition 4.1), for any e > 0, we 
can find a control a 6 = (tt £ ,u £ , X £ ) G U{t) such that the associated dynamics 
Y e ^y satisfy that 

(5.16) V(t, y) + he> E[V(t + h, Y^ y )\. 

For notational simplicity, in what follows we shall drop the superscript " e " 
from each element of the control a £ , when the context is clear. Thus, follow- 
ing the same argument as in the subsolution case, one can show that 

(5.17) eh > Ey* +h (dM*, Y l s ' y ) + [<p](s, *?•»)) ds}. 

Moreover, if we denote As.<5,/i — i su Ps£[t,t+h] \Ys' v ~ u\ ^ ^} f° r the given 
constants e,5,h> 0, then we can find some constant C > such that 

a ( d r t+h ~] 
P{A EAh } = ^E^2 1 \a\Y^^u s )\ 2 d[X%^ 

+ p E \j t \HYs' y ,ir s ,u s )\ 2 dsj 



d r t+h 



(5.18) =p E \Lj t W'iY^^u^dsj 

+ p E \J t \HYs' y ,^u s )\ 2 dsj 
<C(l + \y\ 2 )±h. 
This, together with (5.15) and (5.17), yields, for < h< 5, 

eh > E»y* + \d s <p + ^ 3tUs [<p])(s, Y?'») ds l A c s J - ChP{A eM } 

>h(d s ^(t,y)+ m£_£> ntU [<p](t,y)) - he - C(l + \y\ 2 )^h 2 . 
\ {Tr,u)eu J o 

In other words, it holds that 

(5.19) JU(*,y)+ inf_^ i ^(t,y)<2e + C(l + |2/| 2 )l/ i . 

Finally, first letting h — > and then e — > in (5.19), we derive the desired 
inequality, hence, the result follows. □ 



26 



R. BUCKDAHN, J. MA AND C. RAINER 



6. The uniqueness of the viscosity solution. In this section we discuss the 
uniqueness issue regarding the Hamilton-Jacobi-Bellman equation (5.1). For 
notational simplicity, we assume that d = 1. We should note that, given the 
special form of the HJB equation, its Hamiltonian, defined by 

H(t,y,v,p, S) 

= inf £? n , u [p\{t,y) 

(tt,u) 

= inf \pb(y,ir,u) 

(tt,u) \ 

(6.1) 

a 

+ ^2{ 1 { U i=o}k(. Sal (yi ^ u )^\y^ ^ u )) 
i=i 

-u>*(y,^))/K) 2 }), 

is not continuous in the variables {p, S) . This in fact causes some funda- 
mental difficulties in the uniqueness proof. We shall nevertheless prove a 
uniqueness result under the following extra assumption on the control set 
U. 

(H3) There exists a compact set [/iCM such that: 

(i) 0<£Uv, 

(ii) U = U! or U = {0}UU 1 . 

Remark 6.1. The assumption (H3) amounts to saying that there exist 
positive constants < 5o < C, such that every admissible control u satisfies 

u t ^0 0<(5o<|n t |<C Vt G [0, T], P-a.s. 

Such a restriction is not unusual. For example, in the insurance applications 
the lower bound c could be thought of as the 11 deductible," while the upper 
bound C the "benefit limit" of an insurance policy. 

Our main result of this section is the following theorem. 

Theorem 6.2. Assume (RT)-(H3). Then the value function V : [0,T] x 
]R m — > E defined by (3.4) is the unique viscosity solution of (5.1) among all 
bounded, continuous functions. 
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Proof. We begin with a slight reduction. Let 7 > be any given con- 
stant, and for any function V(t,x), define V(t,x) = e 7 ^ _ ^y(T — t,x). Then, 
it is easy to check that V is a viscosity solution of (5.1) if and only if V is 
a viscosity solution of the following equation: 



(6.2) 



f ^-V(t, x) + jV(t, x) - m(_C^ u V(t, x) = 0, 

ot (w,u)eu 



(t,x) G [0,T] xl™, 
V{Q,x) = e< T g{x), i£l m . 

We now consider two functions V,W G C;,([0,T] x M), with V being the 
subsolution of (6.2) and W the supersolution of (6.2). As usual, we shall 
prove the uniqueness of the solution of (6.2) [hence, of (5.1)] by showing 
that V < W . To this end, let us suppose that 

(6.3) 9= sup (V{t,x)-W{t,x))>0, 

(t,x)e[0,T]xM. 

and look for a contradiction. 

For any e, a > 0, we consider the auxiliary function 

9 6 , a (t,x,8,y) = V(t,x) - W(s,y) - |(|x| 2 + \y\ 2 ) 

ol( \ 1 \ 1 , l2 1 , l2 

— 1 \x-y\ \s-tr. 

2\T-t T-sJ 2e' 1 2e' 1 

Since tends to —00 as t,s —>T or x,y — > 00, one concludes that for any e 
and a there exists (t,x,s,y) = (i e , a , x e , Q , s e , a , y £ ,a) G ([0,r) x K) 2 , such that 

(6.4) *e,a(*, <M,y) = max ^ ea (t,x,s,y) = M Ea - 

([0,T]xM) 2 

Further, by (6.3) and the definition of the supremum, we see that, for all 
rj > 0, there exists a pair (t^,x^), 0<^<T,^6l, such that 

(6.5) F^, x v ) - W{t v , x v )>9- rj/2. 

Combining (6.4) and (6.5), we can find a v such that, for all a G (0, a„), it 
holds that 

M E;Q > V(t v ,x v ) - W(t v ,x v ) - a\x v \ 2 - a- 

(6.6) 



T-t v 



>( 9 - rj/2) -a(\x v \ 2 + jr^j >0- v . 



In other words, if a < a v , then one must have 



V(t,x)-W(s,y) 



\x - y\ 2 + \s — i\ 2 



(6.7) 

ot /, ,9 , .,9v a / 1 1 
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Now, for fixed a > 0, letting e — > 0, we see that 

(6.8) \x-y\ 2 + \s-i\ 2 ^0, \x \ 2 + \y\ 2 < C a . 

Therefore, possibly along a subsequence, still denoted by (t, x, s, y), the fam- 
ily (t,x,s,y) converges to (t a ,x a ,t a ,x a ) for some (t a ,x a ) G [0,T] x R. Con- 
sequently, we have 

K(t, x) - W(s, y) V(t a , x a ) - W{t a , x a ) > 9. 

Here, the last inequality is due to the definition of 9 [cf. (6.3)] and that 

(6.9) limsupf — (\x - y\ 2 + \s- i\ 2 ) + ^(|x| 2 + |y| 2 )) < rj. 

We can now apply Ishii's lemma [7] to obtain that, for all p > 0, there 
exists (X,Y) G R 2m such that 



(6.10) 



and 



t — s a 1 x — y „ \ _i,2,+ Tr/ - A . 
t — sa 1 x — y \ — 1 2 - 



where V ' ' V(i,x) [resp. V ' ' W(i, i)] denotes the "parabolic superjet" 
(resp. "subjets"), as defined in [7]. Choosing p = min(e, ^) < | yields 

(6.11) A + pA 2 <*(^ -! m )+2(a+ l -)( I - ° 

Now, using the definition of a viscosity subsolution (for V) of (5.1) via the 
superjet (resp. supersolution for W via the subjets), we obtain that, for 
all (t,s,x,y) G [0,T] 2 x R 2m and (a,p,S) G V^'+Vfax) [resp. (b,q,S') G 

a + -fV(t,x) 

— inf _( p6(x, 7r, u) 

V(t, x + ua(x, n,u)) — V(t, x) — upa(x, ir, u) 

+ ~2 H*&} 



+ l{ u=0 }{Sa(x,TT,u),a(x,TT,u))^ <0 
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and 

b + jW(s,y) 



inf _( qb(y,ir,u) 



W(s,y + ua(y,Tr,u))-W(s,y)-uqa(y,Tr,u) 
+ 1 {^o} — 2 



+ l {u=0 }(S'a{y,TT,u),a{y,7r,u))^ > 0. 



Indeed, for all (a,p,S) G V^'+Vfax), there is some tp G C7 1 ' 2 ([0,T] x M m ) 
such that V — f < V(t, x) — ip(t, x) and (a,p, S) = (J^(t> x), Vtp(t, x),D 2 {p(t, x)) 
(see [12]). Then the above relation follows from Definition 5.1 with 5 less 
or equal to Sq from Remark 6.1. A symmetrical argument gives the above 
relation for (b,q,S t ) G V^ 2 ''W(t,x). 
Combining the above, we get 

0>a-b + >y(V(t,x) -W(s,y)) 

(6.12) + inf _{qb(y, ir, u) — pb(x, n, u) 

(tt,u)£U 

(6.13) + l {um u- 2 (W(s, y + ua{y, vr, u)) - W(s, y) 

— V(t, x + ua(x, ir, u)) + V{t, x) 
+ u(pa(x, vr, u) - qa{y, ir, it))) 
+ l{ u=0 }{{S'a(y,ir,u),a(y,ir,u)) 

— (Sa(x,ir,u),a(x,7r,u)})}. 
Now, in light of (6.10), we can find two sequences: 

- A t — s a 1 x — y A \ 

= (Sn,ynyb n ,q n , S n ) 



„ i — s a 1 x — y 



as n — > oo. Here, for all n G N, \x n = (a n ,Pn, S n ) G r' 2 ' + V(t n , x n ) [resp. z^ n = 
(b n ,q n ,T n ) eV 1 ' 2 '~W(s n ,y n )]. We now apply (6.12) to each pair (fi n ,u n ) 
and then try to take the limit. Note that the special form of U, thanks to 
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(H3), guarantees that at least along a subsequence, again denoted by itself, 
it holds that 

'^((f^ + (Tf) + * (ii, - ,, ' |y)) 

(6.14) + inf_( — ^(&(x,tt,u) -&(y,vr,u)) 

{ir,u)eu I £ 

+ a(xb(x, 7r, u) + yb(y, it, u)) 

+ 1{ U ^ }U~ 2 \W(s, y + ua{y, w,u)) - W(s, y) 

(6.15) - V(i,x + ua(x,-K,u)) + V(i,x) 

+ u(^—j^-(a(x,-K,u) -a(y,TT,u)) 

+ a(xa(x, 7r, u) + y<j(y, vr, u)) 
+ 1 {u=o}{( Y v{y,' j r,u),(j{y,ir,u)) 

— (Xa(x,TT,u),a(x,TT,u)) 

Now, applying (6.11), we have 

(Ya(y,-K,u),a(y,TT,u)) - (Xa(x,7r,u),a(x,7r,u)) 

= cr T (y, 7r, u)Ya(y, w, u) — cr T (x, it, u)Xa(x, it, u) 



(6.16) 



> --\a(x,x,u) - a(y,TT,u)\ 2 



2^a + ij (\a(x,TT,u)\ 2 + \a(y, vr, u)| 2 ). 



Recalling the definition of (t,x,s,y), we have 

* e , a (£, x, s, y) > * £ , Q (£, ^ + «er(x, 7T, u), s, y + ua(y, ir, u)). 

Using relations (6.14) and (6.16), we get, after some simple straightforward 
calculations, 

> -y(V(t,x) - W{s,y)) - Ca{\ + \x\ + |y|) - C^\x - yf, 

where C is a constant depending only on the bound and Lipschitz constants 
of b and a. Sending e — > and then a — > in the above (taking the "limsup" 
if necessary), and noting (6.9), we obtain that 

rj > 7&. 
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Since rj can be arbitrarily small, we see that this is a contradiction. The 
proof is now complete. □ 

APPENDIX: COUNTEREXAMPLE 

We now construct an example to show that in general equation (2.7) may 
not have a unique solution. Let us consider the case d = 1 and assume that 

the Levy measure is of the form u(dx) = ^zlr x>0 \. 

Suppose that on some probability space we are given a Brownian motion B 
and a Poisson random measure fx whose Levy measure is v. We assume that 
B and /i are independent, define the stopping time S = inf{t > 0,Bf = 1} 

and set u s = ~L[s,+oo){ s )- We shall construct two pairs of coefficients (at, fit) 
and (a' t ,(3' t ), such that the corresponding normal martingales X and X 1 
associated by (2.6) satisfy both (2.7) but are not identical in law. 

To this end, let us set at = lro^t), a' t = — l[o,s](*) and (3t{x) = (3' t (x) = 
lA t (x), where 

A = (0, on [0,5], 

* \ [1, oo), on [S, +oo). 

Clearly, the process N t = Jq J^°° fi(ds de) = li([0, t] x [1, oo)), t > 0, is a stan- 
dard Poisson process independent of B. Now, define Nt = Nt — t, t > 0. 
Applying Proposition 2.1, we see that both processes 

X t = B SAt + N t - N SM , 

X{ = -B S At + N t -N SM , t>0, 

satisfy the structure equation for the above defined process u. We now argue 
that X and X' are not equal in law. Indeed, we write the stopping time S 
as S = inf{i > 0, Xt = 1} and set S' = inf{t >Q,X[ = 1}. Then, it is readily 

seen that X = X' if and only if (X, S) = (X' , S'). 

Now consider the stopped processes Xf = X$At and (X')f' = (X')s'At, 
t>0.HX = X', then X s = (X') s ' as well. But clearly X s is continuous, 
while (X') s ' has jumps on the set {S < S'}, which obviously has a strictly 
positive probability, a contradiction. 
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